Open
Conversation
… into shared_reindexing_service
…ode and upgrade assistant code
…ana into shared_reindexing_service_v2
…ana into shared_reindexing_service_v2
…ana into shared_reindexing_service_v3
Contributor
💔 Build Failed
Failed CI StepsHistory
|
mattkime
added a commit
that referenced
this pull request
Aug 6, 2025
Part of #218984 and #230023 ## Summary Initial creation of ReindexService - extraction of Upgrade Assistant's reindexing functionality into a separate service. Future PRs will provide a meaningful API, rather than just whats needed to maintain current functionality. Summary of actions - Create ReindexService plugin - Create UpgradeAssistant server/common packages to share code between UpgradeAssistant and ReindexService - UpgradeAssistant used an imported singleton class Version to perform version related calculations - now passing the instance as a function argument. Testing this code manually takes a number of steps since its disabled in `main` and will be until the next major version. How to test - 1. add `xpack.upgrade_assistant.ui.enabled: true` to `config/kibana.dev.yml` to enable Upgrade Assistant 2. Because ES is in charge of providing index deprecation info, we'll mock it so Kibana thinks we need to reindex our sample indices and data streams. Mock as shown here - #230021 - note that some of the data stream index references will need to be updated 3. Start kibana 4. Add sample data 5. go to upgrade assistant and try resolving `kibana_sample_data_ecommerce` , `kibana_sample_data_logs`, and `kibana_sample_data_flights` 6. Verify that the indices and data streams have been reindexed. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Ignacio Rivas <rivasign@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
gergoabraham
pushed a commit
to gergoabraham/kibana
that referenced
this pull request
Aug 7, 2025
Part of elastic#218984 and elastic#230023 ## Summary Initial creation of ReindexService - extraction of Upgrade Assistant's reindexing functionality into a separate service. Future PRs will provide a meaningful API, rather than just whats needed to maintain current functionality. Summary of actions - Create ReindexService plugin - Create UpgradeAssistant server/common packages to share code between UpgradeAssistant and ReindexService - UpgradeAssistant used an imported singleton class Version to perform version related calculations - now passing the instance as a function argument. Testing this code manually takes a number of steps since its disabled in `main` and will be until the next major version. How to test - 1. add `xpack.upgrade_assistant.ui.enabled: true` to `config/kibana.dev.yml` to enable Upgrade Assistant 2. Because ES is in charge of providing index deprecation info, we'll mock it so Kibana thinks we need to reindex our sample indices and data streams. Mock as shown here - elastic#230021 - note that some of the data stream index references will need to be updated 3. Start kibana 4. Add sample data 5. go to upgrade assistant and try resolving `kibana_sample_data_ecommerce` , `kibana_sample_data_logs`, and `kibana_sample_data_flights` 6. Verify that the indices and data streams have been reindexed. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Ignacio Rivas <rivasign@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
denar50
pushed a commit
to denar50/kibana
that referenced
this pull request
Aug 8, 2025
Part of elastic#218984 and elastic#230023 ## Summary Initial creation of ReindexService - extraction of Upgrade Assistant's reindexing functionality into a separate service. Future PRs will provide a meaningful API, rather than just whats needed to maintain current functionality. Summary of actions - Create ReindexService plugin - Create UpgradeAssistant server/common packages to share code between UpgradeAssistant and ReindexService - UpgradeAssistant used an imported singleton class Version to perform version related calculations - now passing the instance as a function argument. Testing this code manually takes a number of steps since its disabled in `main` and will be until the next major version. How to test - 1. add `xpack.upgrade_assistant.ui.enabled: true` to `config/kibana.dev.yml` to enable Upgrade Assistant 2. Because ES is in charge of providing index deprecation info, we'll mock it so Kibana thinks we need to reindex our sample indices and data streams. Mock as shown here - elastic#230021 - note that some of the data stream index references will need to be updated 3. Start kibana 4. Add sample data 5. go to upgrade assistant and try resolving `kibana_sample_data_ecommerce` , `kibana_sample_data_logs`, and `kibana_sample_data_flights` 6. Verify that the indices and data streams have been reindexed. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Ignacio Rivas <rivasign@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
NicholasPeretti
pushed a commit
to NicholasPeretti/kibana
that referenced
this pull request
Aug 18, 2025
Part of elastic#218984 and elastic#230023 ## Summary Initial creation of ReindexService - extraction of Upgrade Assistant's reindexing functionality into a separate service. Future PRs will provide a meaningful API, rather than just whats needed to maintain current functionality. Summary of actions - Create ReindexService plugin - Create UpgradeAssistant server/common packages to share code between UpgradeAssistant and ReindexService - UpgradeAssistant used an imported singleton class Version to perform version related calculations - now passing the instance as a function argument. Testing this code manually takes a number of steps since its disabled in `main` and will be until the next major version. How to test - 1. add `xpack.upgrade_assistant.ui.enabled: true` to `config/kibana.dev.yml` to enable Upgrade Assistant 2. Because ES is in charge of providing index deprecation info, we'll mock it so Kibana thinks we need to reindex our sample indices and data streams. Mock as shown here - elastic#230021 - note that some of the data stream index references will need to be updated 3. Start kibana 4. Add sample data 5. go to upgrade assistant and try resolving `kibana_sample_data_ecommerce` , `kibana_sample_data_logs`, and `kibana_sample_data_flights` 6. Verify that the indices and data streams have been reindexed. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Ignacio Rivas <rivasign@gmail.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
mattkime
added a commit
that referenced
this pull request
Oct 8, 2025
…7055) (#237350) # Backport This will backport the following commits from `main` to `8.18`: - [[Upgrade Assistant] Fix privileges for reindexing indices (#237055)](#237055) <!--- Backport version: 10.0.2 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Matthew Kime","email":"matt@mattki.me"},"sourceCommit":{"committedDate":"2025-10-02T13:43:31Z","message":"[Upgrade Assistant] Fix privileges for reindexing indices (#237055)\n\n## Summary\n\nPreviously Upgrade Assistant was checking for `.tasks` index access when\nchecking privs in order to reindex an index. Only the `superuser` role\nprovides access. Further, access is not needed as its been replaced by\nthe tasks api which is available via `cluster: ['manage']`\n\nAdditionally, the saved objects client usage required the `superuser`\nrole since the reindex saved object was hidden and we didn't have a way\nof providing kibana feature privileges for the saved object. The\nsolution is to rely on our our preexisting privilege checks (cluster:\nmanage and 'all' access for the particular indices being reindexed) and\nuse the internal saved object client.\n\nPart of https://github.com/elastic/kibana/issues/237054\n\nTo test -\n\nCreate a role with the following (index names could be more limited and\nit should work)\n```\n{\n \"cluster\": [ \"manage\" ],\n \"index\" : [\n {\n \"names\": [ \"*\" ],\n \"privileges\": [ \"all\" ]\n }\n ]\n}\n```\nassign it to a user. Now try running upgrade assistant and reindexing\nwith that user. It should work.\n\nSimplified testing of upgrade assistant - \nTo test, follow directions here -\nhttps://github.com//pull/228705\nMocked response -\nhttps://github.com//pull/230021/commits/5aab34cdcee2df76d702a058348388a7d10fb73c#diff-f7eb2d7fe666aad1bedcd73d356612d2f74f81c76ba2e8e26b2983b9fb92a661R50\n\n---\n\nRelease note\n\nFixes privilege requirements when reindexing indices via Upgrade\nAssistant. Previously, the \"superuser\" role was required. Now \"cluster:\nmanage\" and \"all\" privileges for the relevant indices are sufficient.","sha":"0250b590f20ac6dcdc5df64ee0a8fd758553957c","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Team:Kibana Management","Feature:Upgrade Assistant","backport:version","v9.2.0","v8.18.8","v8.19.5","v9.0.8","v9.1.5"],"title":"[Upgrade Assistant] Fix privileges for reindexing indices","number":237055,"url":"https://github.com/elastic/kibana/pull/237055","mergeCommit":{"message":"[Upgrade Assistant] Fix privileges for reindexing indices (#237055)\n\n## Summary\n\nPreviously Upgrade Assistant was checking for `.tasks` index access when\nchecking privs in order to reindex an index. Only the `superuser` role\nprovides access. Further, access is not needed as its been replaced by\nthe tasks api which is available via `cluster: ['manage']`\n\nAdditionally, the saved objects client usage required the `superuser`\nrole since the reindex saved object was hidden and we didn't have a way\nof providing kibana feature privileges for the saved object. The\nsolution is to rely on our our preexisting privilege checks (cluster:\nmanage and 'all' access for the particular indices being reindexed) and\nuse the internal saved object client.\n\nPart of https://github.com/elastic/kibana/issues/237054\n\nTo test -\n\nCreate a role with the following (index names could be more limited and\nit should work)\n```\n{\n \"cluster\": [ \"manage\" ],\n \"index\" : [\n {\n \"names\": [ \"*\" ],\n \"privileges\": [ \"all\" ]\n }\n ]\n}\n```\nassign it to a user. Now try running upgrade assistant and reindexing\nwith that user. It should work.\n\nSimplified testing of upgrade assistant - \nTo test, follow directions here -\nhttps://github.com//pull/228705\nMocked response -\nhttps://github.com//pull/230021/commits/5aab34cdcee2df76d702a058348388a7d10fb73c#diff-f7eb2d7fe666aad1bedcd73d356612d2f74f81c76ba2e8e26b2983b9fb92a661R50\n\n---\n\nRelease note\n\nFixes privilege requirements when reindexing indices via Upgrade\nAssistant. Previously, the \"superuser\" role was required. Now \"cluster:\nmanage\" and \"all\" privileges for the relevant indices are sufficient.","sha":"0250b590f20ac6dcdc5df64ee0a8fd758553957c"}},"sourceBranch":"main","suggestedTargetBranches":["8.18","8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/237055","number":237055,"mergeCommit":{"message":"[Upgrade Assistant] Fix privileges for reindexing indices (#237055)\n\n## Summary\n\nPreviously Upgrade Assistant was checking for `.tasks` index access when\nchecking privs in order to reindex an index. Only the `superuser` role\nprovides access. Further, access is not needed as its been replaced by\nthe tasks api which is available via `cluster: ['manage']`\n\nAdditionally, the saved objects client usage required the `superuser`\nrole since the reindex saved object was hidden and we didn't have a way\nof providing kibana feature privileges for the saved object. The\nsolution is to rely on our our preexisting privilege checks (cluster:\nmanage and 'all' access for the particular indices being reindexed) and\nuse the internal saved object client.\n\nPart of https://github.com/elastic/kibana/issues/237054\n\nTo test -\n\nCreate a role with the following (index names could be more limited and\nit should work)\n```\n{\n \"cluster\": [ \"manage\" ],\n \"index\" : [\n {\n \"names\": [ \"*\" ],\n \"privileges\": [ \"all\" ]\n }\n ]\n}\n```\nassign it to a user. Now try running upgrade assistant and reindexing\nwith that user. It should work.\n\nSimplified testing of upgrade assistant - \nTo test, follow directions here -\nhttps://github.com//pull/228705\nMocked response -\nhttps://github.com//pull/230021/commits/5aab34cdcee2df76d702a058348388a7d10fb73c#diff-f7eb2d7fe666aad1bedcd73d356612d2f74f81c76ba2e8e26b2983b9fb92a661R50\n\n---\n\nRelease note\n\nFixes privilege requirements when reindexing indices via Upgrade\nAssistant. Previously, the \"superuser\" role was required. Now \"cluster:\nmanage\" and \"all\" privileges for the relevant indices are sufficient.","sha":"0250b590f20ac6dcdc5df64ee0a8fd758553957c"}},{"branch":"8.18","label":"v8.18.8","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.0","label":"v9.0.8","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/237334","number":237334,"state":"OPEN"},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/237326","number":237326,"state":"OPEN"},{"url":"https://github.com/elastic/kibana/pull/237325","number":237325,"branch":"9.2","state":"OPEN"}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
mattkime
added a commit
that referenced
this pull request
Oct 8, 2025
…7055) (#237340) # Backport This will backport the following commits from `main` to `8.19`: - [[Upgrade Assistant] Fix privileges for reindexing indices (#237055)](#237055) <!--- Backport version: 10.0.2 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Matthew Kime","email":"matt@mattki.me"},"sourceCommit":{"committedDate":"2025-10-02T13:43:31Z","message":"[Upgrade Assistant] Fix privileges for reindexing indices (#237055)\n\n## Summary\n\nPreviously Upgrade Assistant was checking for `.tasks` index access when\nchecking privs in order to reindex an index. Only the `superuser` role\nprovides access. Further, access is not needed as its been replaced by\nthe tasks api which is available via `cluster: ['manage']`\n\nAdditionally, the saved objects client usage required the `superuser`\nrole since the reindex saved object was hidden and we didn't have a way\nof providing kibana feature privileges for the saved object. The\nsolution is to rely on our our preexisting privilege checks (cluster:\nmanage and 'all' access for the particular indices being reindexed) and\nuse the internal saved object client.\n\nPart of https://github.com/elastic/kibana/issues/237054\n\nTo test -\n\nCreate a role with the following (index names could be more limited and\nit should work)\n```\n{\n \"cluster\": [ \"manage\" ],\n \"index\" : [\n {\n \"names\": [ \"*\" ],\n \"privileges\": [ \"all\" ]\n }\n ]\n}\n```\nassign it to a user. Now try running upgrade assistant and reindexing\nwith that user. It should work.\n\nSimplified testing of upgrade assistant - \nTo test, follow directions here -\nhttps://github.com//pull/228705\nMocked response -\nhttps://github.com//pull/230021/commits/5aab34cdcee2df76d702a058348388a7d10fb73c#diff-f7eb2d7fe666aad1bedcd73d356612d2f74f81c76ba2e8e26b2983b9fb92a661R50\n\n---\n\nRelease note\n\nFixes privilege requirements when reindexing indices via Upgrade\nAssistant. Previously, the \"superuser\" role was required. Now \"cluster:\nmanage\" and \"all\" privileges for the relevant indices are sufficient.","sha":"0250b590f20ac6dcdc5df64ee0a8fd758553957c","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Team:Kibana Management","Feature:Upgrade Assistant","backport:version","v9.2.0","v8.18.8","v8.19.5","v9.0.8","v9.1.5"],"title":"[Upgrade Assistant] Fix privileges for reindexing indices","number":237055,"url":"https://github.com/elastic/kibana/pull/237055","mergeCommit":{"message":"[Upgrade Assistant] Fix privileges for reindexing indices (#237055)\n\n## Summary\n\nPreviously Upgrade Assistant was checking for `.tasks` index access when\nchecking privs in order to reindex an index. Only the `superuser` role\nprovides access. Further, access is not needed as its been replaced by\nthe tasks api which is available via `cluster: ['manage']`\n\nAdditionally, the saved objects client usage required the `superuser`\nrole since the reindex saved object was hidden and we didn't have a way\nof providing kibana feature privileges for the saved object. The\nsolution is to rely on our our preexisting privilege checks (cluster:\nmanage and 'all' access for the particular indices being reindexed) and\nuse the internal saved object client.\n\nPart of https://github.com/elastic/kibana/issues/237054\n\nTo test -\n\nCreate a role with the following (index names could be more limited and\nit should work)\n```\n{\n \"cluster\": [ \"manage\" ],\n \"index\" : [\n {\n \"names\": [ \"*\" ],\n \"privileges\": [ \"all\" ]\n }\n ]\n}\n```\nassign it to a user. Now try running upgrade assistant and reindexing\nwith that user. It should work.\n\nSimplified testing of upgrade assistant - \nTo test, follow directions here -\nhttps://github.com//pull/228705\nMocked response -\nhttps://github.com//pull/230021/commits/5aab34cdcee2df76d702a058348388a7d10fb73c#diff-f7eb2d7fe666aad1bedcd73d356612d2f74f81c76ba2e8e26b2983b9fb92a661R50\n\n---\n\nRelease note\n\nFixes privilege requirements when reindexing indices via Upgrade\nAssistant. Previously, the \"superuser\" role was required. Now \"cluster:\nmanage\" and \"all\" privileges for the relevant indices are sufficient.","sha":"0250b590f20ac6dcdc5df64ee0a8fd758553957c"}},"sourceBranch":"main","suggestedTargetBranches":["8.18","8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/237055","number":237055,"mergeCommit":{"message":"[Upgrade Assistant] Fix privileges for reindexing indices (#237055)\n\n## Summary\n\nPreviously Upgrade Assistant was checking for `.tasks` index access when\nchecking privs in order to reindex an index. Only the `superuser` role\nprovides access. Further, access is not needed as its been replaced by\nthe tasks api which is available via `cluster: ['manage']`\n\nAdditionally, the saved objects client usage required the `superuser`\nrole since the reindex saved object was hidden and we didn't have a way\nof providing kibana feature privileges for the saved object. The\nsolution is to rely on our our preexisting privilege checks (cluster:\nmanage and 'all' access for the particular indices being reindexed) and\nuse the internal saved object client.\n\nPart of https://github.com/elastic/kibana/issues/237054\n\nTo test -\n\nCreate a role with the following (index names could be more limited and\nit should work)\n```\n{\n \"cluster\": [ \"manage\" ],\n \"index\" : [\n {\n \"names\": [ \"*\" ],\n \"privileges\": [ \"all\" ]\n }\n ]\n}\n```\nassign it to a user. Now try running upgrade assistant and reindexing\nwith that user. It should work.\n\nSimplified testing of upgrade assistant - \nTo test, follow directions here -\nhttps://github.com//pull/228705\nMocked response -\nhttps://github.com//pull/230021/commits/5aab34cdcee2df76d702a058348388a7d10fb73c#diff-f7eb2d7fe666aad1bedcd73d356612d2f74f81c76ba2e8e26b2983b9fb92a661R50\n\n---\n\nRelease note\n\nFixes privilege requirements when reindexing indices via Upgrade\nAssistant. Previously, the \"superuser\" role was required. Now \"cluster:\nmanage\" and \"all\" privileges for the relevant indices are sufficient.","sha":"0250b590f20ac6dcdc5df64ee0a8fd758553957c"}},{"branch":"8.18","label":"v8.18.8","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.0","label":"v9.0.8","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/237334","number":237334,"state":"OPEN"},{"branch":"9.1","label":"v9.1.5","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/237326","number":237326,"state":"OPEN"},{"url":"https://github.com/elastic/kibana/pull/237325","number":237325,"branch":"9.2","state":"OPEN"}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Summarize your PR. If it involves visual changes include a screenshot or gif.
Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
release_note:breakinglabel should be applied in these situations.release_note:*label is applied per the guidelinesbackport:*labels.Identify risks
Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.